Transformation of random vectors

Functions of random variables

Definition 1. Borel measurable sets on a space \(\R^n\) is denoted by \(\sB(\R^n)\) and generated by the collection \((\pi_i^{-1}(-\infty,x]: x \in \R, i \in [n])\). A function \(g: \R^n\to \R^m\) is called Borel measurable function, if \(g^{-1}(B_m) \in \sB(\R^n)\) for any \(B_m \in \sB(\R^m)\).

Proposition 2. Consider a random variable \(X: \Omega \to \R\) defined on the probability space \((\Omega, \sF, P)\). Suppose \(g: \R \to \R\) is function such that \(g^{-1}(-\infty, x] \in \sB(\R)\), then \(g(X)\) is a random variable.

Proof. Proof. We represent \(g(X)\) by a map \(Y: \Omega \to \R\) such that \(Y(\omega) \triangleq (g \circ X)(\omega)\) for all outcomes \(\omega \in \Omega\). We further check that for any half open set \(B_x = (-\infty,x]\), we have \(Y^{-1}(B_x) = (X^{-1}\circ g^{-1})(B_x)\). Since \(g^{-1}(B_x) \in \sB(\R)\), it follows that \(Y^{-1}(B_x) \in \sF\) by the definition of random variables. ◻

Example 3 (Monotone function of random variables). Let \(g:\R \to \R\) be a monotonically increasing function, then \(g^{-1}(-\infty, x] %= (g^{-1}(-\infty), g^{-1}(x)] \in \sB(\R)\) for all \(x \in \R\). Consider a random variable \(X: \Omega \to \R\) defined on the probability space \((\Omega, \sF, P)\), then \(Y \triangleq g(X)\) is a random variable with distribution function Here, \(g^{-1}(y)\) is the functional inverse, and not inverse image as we have been seeing typically. We can think \(g^{-1}(y) = g^{-1}\set{y}\), though this inverse image has at most a single element since \(g\) is monotonically increasing.

Example 4. Consider a positive random variable \(X: \Omega \to \R_+\) defined on a probability space \((\Omega,\sF,P)\). Let \(g: \R_+ \to \R_+\) be such that \(g(x) = e^{-\theta x}\) for all \(x \in \R_+\) and some \(\theta > 0\). Then, \(g\) is monotonically decreasing in \(X\) and \(x = g^{-1}(y) = -\frac{1}{\theta}\ln y\). This implies that \(g^{-1}(-\infty, y] = [-\frac{1}{\theta}\ln y, \infty) \in \sB(\R_+)\) for all \(y \in \R_+\). Thus \(g\) is a measurable function, and \(Y = g(X)\) is a random variable.

Proposition 5 (Independence of function of random variables). Let \(g:\R \to \R\) and \(h: \R \to \R\) be functions such that \(g^{-1}(-\infty, x]\) and \(h^{-1}(-\infty, x]\) are Borel sets for all \(x \in \R\). Consider independent random variables \(X\) and \(Y\) defined on the probability space \((\Omega, \sF, P)\), then \(g(X)\) and \(h(Y)\) are independent random variables.

Proof. Proof. For any \(u,v \in \R\), we can define inverse images \(A_g(u) \triangleq g^{-1}(-\infty,u]\) and \(A_h(v) \triangleq h^{-1}(\infty, v]\). Since \(g,h\) are Borel measurable, we have \(A_g(u), A_h(v) \in \sB(\R)\). We can write the following outcome set equality for the joint event Since \(X\) and \(Y\) are independent random variables, it follows that \(X^{-1}(A_g(u))\) and \(Y^{-1}(A_h(v))\) are independent events, and the result follows. ◻

Function of random vectors

Proposition 6. Consider a random vector \(X: \Omega \to \R^n\) defined on the probability space \((\Omega, \sF, P)\), and a Borel measurable function \(g: \R^n \to \R^m\) such that \(A_g(y) \triangleq \cap_{j=1}^m\set{x \in \R^n: g_j(x) \le y_j} \in \sB(\R^n)\) for all \(y \in \R^m\). Then, \(g(X): \Omega \to \R^m\) is a random vector. The joint distribution function \(F_Y: \R^m \to [0,1]\) for the vector \(Y\triangleq g(X)\) is given by

Example 7 (Sum of random variables). For a random vector \(X: \Omega \to \R^n\) defined on a probability space \((\Omega, \sF, P)\). Define an addition function \(+: \R^n \to \R\) such that \(+(x) = \sum_{i=1}^nx_i\) for any \(x \in \R^n\). We can verify that \(+\) is a Borel measurable function and hence \(Y = +(X) = \sum_{i=1}^nX_i\) is a random variable. When \(n=2\) and \(X\) is a continuous random vector with density \(f_X: \R^2 \to \R_+\), we can write By applying a change of variable \((x_1, t)= (x_1, x_1 + x_2)\) and changing the order of integration, we see that When \(Y\) is a continuous random vector, we can write When \(X: \Omega \to \R^2\) is an independent vector, then \(f_X(x) = f_{X_1}(x_1)f_X(x_2)\) for all \(x\in\R^2\). Therefore, the density of the sum \(X_1+X_2\) is given by where \(\ast: \R^{\R} \times \R^{\R} \to \R^{\R}\) is the convolution operator.

Theorem 8. For a continuous random vector \(X:\Omega \to \R^m\) defined on the probability space \((\Omega, \sF, P)\) with density \(f_X: \R^m \to \R_+\) and an injective and smooth Borel measurable function \(g: \R^m \to \R^m\), such that \(Y=g(X)\) is a continuous random vector. Then the density of random vector \(Y\) is given by where \(x=g^{-1}(y)\) and \(J(y)= (J_{ij}(y) \triangleq \frac{\partial y_j}{\partial x_i}: i,j\in [m])\) is the Jacobian matrix.

Proof. Proof. For an injective map \(g: \R^m \to \R^m\) we have \(\set{x} = g^{-1}\set{y}\) for any \(y \in g(\R^m)\). Further, since \(g\) is smooth, we have \(dy = J(y)dx + o(\abs{dx})\), and thus Defining set \(dB(y) \triangleq \set{w \in \R^m: y_j \le w_j \le y_j+dy_j}\), we observe that for any continuous random vector \(Y: \Omega \to \R^m\), we have We get the result by combining [eqn:Jacobian] and [eqn:density]. ◻

Example 9 (Sum of random variables). Suppose that \(X: \Omega \to \R^2\) is a continuous random vector and \(Y_1 = X_1 + X_2\). Let us compute \(f_{Y_1}(y_1)\) using the above theorem. Let us define a random vector \(Y: \Omega \to \R^2\) such that \(Y = (X_1+X_2, X_2)\) so that \(\abs{J(y)} = 1\). This implies, \(f_{Y}(y) = f_{X}(x)\). Thus, we may compute the marginal density of \(Y_1\) as, If \(X\) is an independent random vector, then where \(*\) represents convolution.